Grubb’s Econometrics


Grubb’s econometric evidence consists of five tables of results characterizing the time series behavior of early colonial and state prices and exchange rates in such a way as to suggest that prices and exchange rates behaved benignly before the mid-1790s, when colonial/state-issued paper money predominated, and badly thereafter, due to the transition to the dollar.


Our position is that these econometric exercises serve no useful purpose, and do not, and could not, establish Grubb’s proposition, even had the econometrics been performed flawlessly (which, in our view, it was not). We hold this opinion for the following reasons.


First, the tables for the most part are devoted to comparing the behavior of macroeconomic time series before and after the mid-1790s. Whenever Grubb finds “statistically significant” structural breaks in the mid-1790s, he attributes them to the transition to the dollar. Our view, which is developed at length in the published version of the paper, is that the only aspect of the “transition to the dollar” that might plausibly have occurred in the mid-1790s was a simple change in the unit of account -- the more substantive changes (most notably, the emergence of banks and the disappearance of colony/state-issued bills of credit from circulation) generally came earlier. That a mere change in the unit of account could drastically alter the macroeconomic behavior of the American economy is implausible; if a discontinuity exists in the mid-1790s, it must be due to other factors.


The second kind of evidence Grubb presents are various unit root tests. For example, for the period after the transition, Grubb cannot reject a unit root in purchasing power parity relationships; he takes this finding as evidence of macroeconomic instability due to the new monetary arrangements. Other unit root tests Grubb performed on “pre-transition” data usually reject unit roots, which he interprets as evidence of greater stability in the earlier era. There are at least two difficulties with this line of argument. Despite the fact that many time series practitioners use the presence or absence of unit roots as a proxy for judging whether economic relationships are unstable or not, the connection is a very dubious one. McCallum (1993, pp. 34-38) presents a persuasive case, in our view, that the presence or absence of unit roots in such economic data is essentially an irrelevant consideration.


The second difficulty is that most of Grubb’s data series are extremely short, frequently less than two dozen observations. Even in samples of size 50, Dickey-Fuller tests have meager power against alternatives in the general vicinity of a unit root (Dickey, Bell, and Miller, 1986, p. 18). Even the size of Dickey-Fuller tests can be wildly inaccurate if the true series contains a moving average component (Schwert, 1989). Moreover, Grubb -- unavoidably -- had to perform preliminary tests to choose his lag lengths and to decide whether to include deterministic trends, and these tests had to be performed using the same scanty data. All Grubb’s statistical findings, weak as they would be if these factors were known a priori, are subject to further qualification because their accuracy is contingent upon the correct determination of lag lengths and trends. In short, we think it impossible to draw meaningful conclusions about unit roots from such short time series and also consider the entire unit root question to be an irrelevant one, for the reasons detailed by McCallum (1993).


Given our conviction that Grubb’s econometric findings are essentially irrelevant, we declined – to the disappointment of our referees – to enter into an a full-fledged econometric duel. However, we recognize that many readers may have a different view of the methodological issues, and hesitate to discountenance a paper utilizing modern econometrics in favor of an alternative view based entirely on narrative history. Therefore, we conclude with a few comments on Grubb’s econometrics.


Grubb’s Table 3 (2003, p. 1793) is arguably the most important in his paper, for it tests for a structural break in the time series behavior of prices coincident with the transition to the dollar. We believe that Grubb’s conclusions are highly sensitive to the precise timing of the structural break, that the dates he used were quite favorable to his case, and that the choice of dates was debatable.


According to Grubb’s notes: “For U.S. states, D is a structural break variable representing the transition from prices quoted in state currency to prices quoted in the U.S. dollar. It is coded as one for years greater than 1792 in Massachusetts, 1794 for New York and Pennsylvania, and 1795 for Maryland, Virginia, and South Carolina, and zero otherwise.” The dating of the structural break is of course crucial because placing the break in different years produces very different econometric results. At one point, Grubb writes (2003, p. 1782) that the behavior of prices, as revealed in his Figure 1 and Figure 2, can be used to “identify when markets shifted from state currency to U.S. dollars.” Footnote


Had the dates been determined in this manner, there would be a serious issue with pretest bias. Elsewhere though, Grubb states that he dated the transitions by determining the date at which the unit of account changed. Only in the case of Pennsylvania, however, does he detail the source consulted to determine the date, and in Pennsylvania, his own sources suggest the transition ought to be placed in 1796, not 1794. When the equation is estimated using 1796 and not 1794, the t-statistic on the structural break dummy falls from 5.59 to 1.29 (although an interaction term with the trend remains significant).


It is also difficult to see how Grubb settled on 1792 as the date of the transition in Massachusetts. Winifred B. Rothenberg’s article (1979), his source for the data, says nothing about when the unit of account changed in Massachusetts, and since Rothenberg’s price index was constructed using data gleaned from two dozen account books, a diary, and 350 probate inventories, Grubb could scarcely have relied on her original sources for his dating. The one secondary source cited by Grubb that does date changes in the unit of account is Cole (1938, p.115), which reports Boston wheat prices were still being quoted in shillings in 1798. Moreover, in 1795 Massachusetts passed a law dictating that dollars and cents would be the money of account after 1 September 1795, a measure that would have been unnecessary had the unit of account changed in 1792 (Massachusetts, 1795, chap. 18, pp. 452-3). Placing the Massachusetts structural break dummy in 1792, as Grubb does, results in a t-statistic of 3.59, but it drops to 2.68 with the dummy in 1793, 0.45 in 1794, -0.29 in 1795, -0.47 in 1796, and -0.81 in 1797.


Cole (1938) does not support Grubb’s transition dates for Pennsylvania, New York, or South Carolina either. Philadelphia prices were almost all quoted in dollars as early as 1792. New York prices were quoted in pounds, shillings, and pence until 1796. Charleston prices were quoted in pounds, shillings and pence until 1803. (Cole has no data on Virginia or Maryland.) Using Cole’s dating for Pennsylvania, New York and South Carolina reduces Pennsylvania and South Carolina’s structural breaks to statistical insignificance, leaving only New York (where the date shifts only a year) as statistically significant. Thus Grubb’s conclusions regarding the statistical significance of the structural break are sensitive to his choice of break years, and his choices are debatable.


Grubb’s Table 4 (2003, p. 1794) seeks to establish that the inflation rate was more variable after the mid-1790s than earlier. To this end, Grubb presents point estimates of standard deviations of the inflation rate, along with estimated standard errors for each state in each of the following three periods: 1750-1775, 1785-1792, and 1797-1811. The observed pair-wise differences in standard deviations across periods are declared to be statistically significant at the 10% level in all cases but one.


The distribution theory underlying the claim of statistical significance, however, is flawed. The first, comparatively minor problem, is that the standard errors of the sample standard deviations are produced using a bootstrap method that assumes inflation rates are statistically independent over time. Grubb’s Table 1 (2003, p. 1784) is devoted to establishing the time series properties of the inflation rate, and he never finds prices to be a random walk, implying his bootstrap estimates of the standard errors are doubtful. This, however, is but a quibble compared to another error in the same table. If one examines Table 4 carefully, one can see that Grubb reports the standard deviation of the Massachusetts exchange rate to be 0.0791 in 1785-1792, with a standard error of .0147, while in 1797-1811, the standard deviation is reported to be 0.0956, with a standard error of 0.0184. Those point estimates differ by only 0.0162, less than one standard error of the estimate for 1797-1811 alone. The difference, Grubb says, is statistically significant at the 10% level, although that is clearly impossible.


Grubb misapplied a t-test designed for testing the difference of population means; consequently, all the comparisons in Table 4 are seriously biased towards rejection of the null. In an email, Grubb explained that the calculation was performed in this manner:

ole.gif


An argument might be made for the asymptotic normality of a statistic similar to this one, but one would have to omit n1 and n2 from the calculation.


If one were willing to assume that the annual inflation rates were statistically independent over time (an assumption Grubb makes implicitly in his bootstrapping) and that the inflation rates were draws from independent normally distributed populations (a requirement for the t-test on means Grubb misapplies) we could fix this serious error and perform a correct test using the fact that the ratio of the sample variances would have an F distribution under the null hypothesis of equal population variances. In the comparison of 1797-1811 with 1785-1792, where Grubb finds all the differences save one (Pennsylvania) statistically significant, the F-test (using alpha = .10, as Grubb does) finds all the differences save Maryland statistically insignificant. In the comparison of 1797-1811 with 1750-1775, Grubb finds each of the differences to be significant, although two of the differences (Massachusetts and South Carolina) go in the “wrong direction” in the sense that variability was greater in colonial than in federal times. Comparing 1797-1811 to 1750-1775 using the F distribution and alpha = .10, New York and South Carolina become insignificant. Using the more conventional alpha = .05 would also make the cases of Massachusetts and Maryland insignificant.


Even these tests are problematic, because the F test is known to be sensitive to departures from normality, and because Grubb provides considerable evidence suggesting that these annual inflation rates are serially correlated over time. Some kind of test could probably be constructed to overcome these difficulties, but doing so would require considerable effort on our part, and would only serve to persuade the reader that we viewed the matter to be one of great importance, which we do not. After all, once the lack of independence is recognized, even the appropriate null hypothesis is unclear – should one test whether the unconditional variances of inflation are equal, or whether the innovations in the variances of inflation are equal? Therefore we shall not carry the matter further than to conclude that the results reported in Table 4 are incorrect and that they greatly exaggerate the evidence suggesting inflation was more variable after 1797 than before.


Grubb’s Table 5 (2003, p. 1795) is intended to demonstrate that after the mid-1790s estimates of a purchasing power parity relationship between each of the American states and England show purchasing power parity failing, in the sense that one cannot reject the presence of a unit root. As a general matter, accepting any null hypothesis does not prove the null hypothesis is true. That observation is especially apt in this instance; for a sample sizes of 50, the Dickey-Fuller test is known to have low power against alternatives anywhere near the unit circle, and Grubb’s sample sizes never exceed 17 (Dickey, Bell, and Miller, 1986, p. 18). All the tests in this table, however, should be disregarded, because Grubb used an inappropriate exchange rate series. Grubb found his exchange rate data in Officer (1996), but Officer had processed the data in various ways, most notably by making sizable adjustments (Officer, 1996, p. 77, table 6.5) to correct the exchange rate for the inflation in England during the period of the paper pound (1797-1815). Such adjustments are inappropriate for a test of purchasing power parity, but Grubb failed to note them and back them out. Footnote


We mention this as yet another illustration of the weakness of Grubb’s econometrics, not because using the correct series results in striking rejections of unit roots. Like McCallum (1993) we believe the presence of unit roots is irrelevant to the issue at hand, and like most sensible statisticians, we recognize that accepting a null hypothesis, particularly when using a test practically devoid of power against plausible alternatives, cannot be construed as proof that the null hypothesis is true.